Method and system for displaying video from a video game console

ABSTRACT

A system for displaying video from a video game console is provided. The system includes an audio and video data conversion device in communication with the video game console. Additionally, a computer in communication with the audio and video data conversion device is included. The system also includes a display panel in communication with the computer, whereby the display panel is capable of displaying the video from the video game console. A method for displaying the video from the video game console also is described.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of application Ser. No.11/059,972, filed on Feb. 16, 2005, the disclosure of this applicationis incorporated herein by reference.

BACKGROUND OF THE INVENTION

Video game consoles are usually connected to regular television sets forvisual and sounds effects. When compared to a computer monitor, atelevision set typically has lower image quality since the televisionset displays video at lower resolution. However, video game consoles donot interface with computer monitors through computers because a userwill typically experience audio latency whereby the audio and video arenot synchronized. In other words, the audio may lag behind or lead thevideo by a few seconds.

Audio latency may be caused by the use of different clock frequencies byan audio and video data conversion device and by the computer. Inparticular, a first frequency used within the audio and video dataconversion device is typically different from a second frequency of anaudio capture clock at which an audio renderer within the computer sendsaudio data to an audio encoder. The audio latency can cause a noticeabledelay between a user's input actions through the video game console andresultant audio and video outputted from a computer.

As a result, there is a need to provide methods and systems for reducingaudio latency and for displaying video from a video game console.

SUMMARY OF THE INVENTION

Broadly speaking, the present invention fills these needs by providingmethods and systems for displaying video from a video game console. Itshould be appreciated that the present invention can be implemented innumerous ways, including as a method, a system, or a device. Severalinventive embodiments of the present invention are described below.

In accordance with a first aspect of the present invention, a system fordisplaying video from a video game console is provided. The systemincludes an audio and video data conversion device in communication withthe video game console. Additionally, a computer in communication withthe audio and video data conversion device is included. The system alsoincludes a display panel in communication with the computer, whereby thedisplay panel is capable of displaying the video.

In accordance with a second aspect of the present invention, a methodfor displaying video from a video game console is provided. In thismethod, the video from an audio and video data conversion device isreceived at a computer. The video is fed to the audio and videoconversion device by the video game console. Thereafter, the video isoutputted to a display panel.

In accordance with a third aspect of the present invention, a method fordisplaying video from a video game console is provided. In this method,analog video from the video game console is converted to digitizedvideo. After the conversion, the digitized video is outputted to acomputer.

Other aspects and advantages of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, illustrating by way of example the principles ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be readily understood by the followingdetailed description in conjunction with the accompanying drawings, andlike reference numerals designate like structural elements.

FIG. 1 is a simplified block diagram of a system for displaying videofrom a video game console, in accordance with one embodiment of thepresent invention.

FIG. 2 is a more detailed schematic diagram of the audio and video dataconversion device shown in FIG. 1, in accordance with one embodiment ofthe present invention.

FIG. 3 is an alternative embodiment to the audio and video dataconversion device of FIG. 2.

FIG. 4 is a flowchart diagram of a high level overview for displayingvideo from a video game console, in accordance with one embodiment ofthe present invention.

FIG. 5 is a simplified block diagram of a system for rendering analogaudio signals, in accordance with one embodiment of the presentinvention.

FIG. 6 is a more detailed block diagram of the operating system shown inFIG. 5, in accordance with one embodiment of the present invention.

FIG. 7 is a flowchart diagram of a high level overview of a method forreducing audio latency when executing program instructions forprocessing audio data, in accordance with one embodiment of the presentinvention.

FIG. 8 is a flowchart diagram of a more detailed method for reducingaudio latency when executing program instructions for processing audiodata, in accordance with one embodiment of the present invention.

FIG. 9 is a flowchart diagram of a detailed method for implementing anembodiment of the present invention in Microsoft Windows, in accordancewith one embodiment of the present invention.

FIG. 10 is a schematic diagram of a main graphical user interface (GUI)associated with a video game console display application, in accordancewith one embodiment of the present invention.

DETAILED DESCRIPTION

An invention is described for hardware implemented methods and systemsfor displaying video from a video game console and reducing audiolatency. It will be obvious, however, to one skilled in the art, thatthe present invention may be practiced without some or all of thesespecific details. In other instances, well known process operations havenot been described in detail in order not to unnecessarily obscure thepresent invention.

A. Display of Graphics from Video Game Console

FIG. 1 is a simplified block diagram of a system for displaying videofrom a video game console, in accordance with one embodiment of thepresent invention. As shown in FIG. 1, system includes video gameconsole 102, audio and video data conversion device 150, and computer104. Video game console 102 is a computing device primarily designed toplay video games. Computer 104 can include any suitable computingdevice, and display panel 101 and speaker 172 can be attached to thecomputer to output video and audio, respectively. Display panel 101 is aperipheral device capable of displaying still or moving image. Examplesof display panel 101 include cathode ray tube (CRT) displays, liquidcrystal displays (LCD), plasma displays, video projectors, etc. Speaker172 can include any suitable device that converts electrical signalsinto sounds.

As shown in FIG. 1, audio and video data conversion device 150 is incommunication with video game console 102 and computer 104. Video gameconsole 102 feeds analog audio and/or video to audio and video dataconversion device 150. Video game console can output analog audio andvideo through any suitable ports. Exemplary ports include RadioCorporation of America (RCA) composite jacks, S-Video ports, televisionradio frequency (RF) ports, etc. As will be explained in more detailbelow, audio and video data conversion device 150 receives the analogaudio and/or video from video game console 102 and digitizes the audioand/or video. In one embodiment, the digitized audio and/or video isthen outputted to computer 104 in raw form. In other words, thedigitized audio and/or video is directly outputted to computer 104. Inanother embodiment, the digitized audio and/or video is furthercompressed and then outputted to computer 104.

Computer 104 receives the audio and/or video from audio and video dataconversion device 150 through a computer interface and outputs the videoand audio to display panel 101 for display and to speaker 172,respectively. In one embodiment, before the audio and/or video isoutputted, computer 104, as will be explained in more detail below, canprocess the audio to reduce audio latency and adjust properties of thevideo. Accordingly, audio and video data conversion device 150 allowsdisplay panel 101 and speaker 172 attached to computer 104 to be used todisplay video and to render audio, respectively, outputted from videogame console 102.

FIG. 2 is a more detailed schematic diagram of the audio and video dataconversion device shown in FIG. 1, in accordance with one embodiment ofthe present invention. As shown in FIG. 2, audio and video dataconversion device 150 includes audio and video processing circuitry 106and device controller 108. Audio and video processing circuitry 106digitizes analog audio 154 and/or analog video 110 received from a videogame console. Audio and video processing circuitry 106 can include videodecoder 107 that converts analog video 110 to a digital format, such asYCbCr signals or YUV signals. An exemplary video decoder 107 is theConexant CX25840 chip. It should be appreciated that video game consolesare used worldwide and therefore are designed to cater to differentvideo standards, such as phase alternation line (PAL), NationalTelevision System Committee (NTSC), and séquentiel couleur avec mémoire(SÉCAM). In one embodiment, video decoder 107 can decode these videostandards automatically.

Audio and video processing circuitry 106 can additionally include audiodecoder 162 (e.g., Asahi Kasei Microsystem's AKM5357 chip, which is a24-bit 2-channel A/D converter) that converts analog audio 154 to adigital format. For example, audio decoder 162 can digitize analog audio154 received from a video game console into a sixteen bit or higherstereo serial data bit stream. In the embodiment of FIG. 2, audiodecoder 162 outputs the digitized audio to video decoder 107 for packingwith the digitized video by time multiplexing the digitized audio intothe digitized video stream. Audio and video processing circuitry 106then outputs the combined audio and video data to computer 104 throughdevice controller 108. It should be appreciated that audio and videodata conversion device 150 can adopt to a variety of interfaces, such asUniversal Serial Bus (USB), FireWire, Peripheral Component Interconnect(PCI), PCI-X, PCI Express, Cardbus, etc. Depending on the type ofinterface used, device controller 108 can include any suitablecontroller for communicating with computer 104. Exemplary devicecontroller 108 includes a USB controller (e.g., Cypress's CY7C68013Achip), a FireWire controller, a PCI controller, a PCI-X controller, aPCI Express controller, etc. As shown in FIG. 2, audio and video dataconversion device 150 is separate from or external to the video gameconsole and computer 104. However, depending on the type of interfaceused, audio and video data conversion device 150 can also be integratedwithin computer 104. Furthermore, it should be appreciated that videodecoder 107, audio decoder 162, and device controller 108 can beintegrated into a single chip.

FIG. 3 is an alternative embodiment to the audio and video dataconversion device of FIG. 2. In addition to audio and video processingcircuitry 106 and device controller 108, audio and video data conversiondevice 150 of FIG. 3 additionally includes television (TV) tuner 111,light-emitting diode (LED) 109, and non-volatile storage chip 112. TVtuner 111 allows computer 104 to additionally receive television signalsby converting the television signals into a video stream and outputtingthe video stream to video decoder 107.

It should be appreciated that computer 104 can use device controller 108to control operations of the various sub-modules (e.g., video decoder107, audio decoder 162, LED 109, etc.) of audio and video dataconversion device 150. Computer 104 can send commands to devicecontroller 108 and, in turn, the device controller extracts the commandsfor processing, execution, and control of the sub-modules. In oneembodiment, device controller 108 may store the commands from computer104 to non-volatile storage chip 112 (e.g., Electrically-ErasableProgrammable Read-Only Memory (EEPROM)) for subsequent retrieval andexecution.

FIG. 4 is a flowchart diagram of a high level overview for displayingvideo from a video game console, in accordance with one embodiment ofthe present invention. Starting in operation 113, a computer receivesvideo from an audio and video data conversion device. The video is fedto the audio and video conversion device by a video game console. Itshould be appreciated that in another embodiment, the computer canadditionally receive audio from the audio and video data conversiondevice, where the audio is fed to the audio and video conversion deviceby the video game console. Accordingly, in this embodiment, the audioand video data conversion device can simultaneously feed both the videoand audio from the video game console to the computer. As will beexplained in more detail below, embodiments of the present invention cankeep the latency between audio and video effects to a minimum, therebypreserving a real-time gaming effect.

After the computer receives the video, the computer outputs the video inoperation 114 to a display panel that is in communication with thecomputer for display. As will be explained in more detail below, beforethe computer outputs the video, the computer can adjust properties ofthe video and store the video onto a memory of the computer.

B. Audio Latency Reduction

The embodiments described herein provide method and system for reducingaudio latency when executing program instructions for processing audiodata. In one embodiment, an amount of audio data stored in an audiobuffer is determined, and the amount is compared with a top thresholdvalue and a bottom threshold value. It should be noted that the terms“audio data,” “audio,” and “audio signal” may be used interchangeably.As will be explained in more detail below, an audio data feed to anaudio renderer is adjusted incrementally such that the amount is betweenthe top threshold value and the bottom threshold value. By keeping theamount of audio data stored in the audio buffer within the top thresholdvalue and the bottom threshold value, audio latency is reduced oraltogether eliminated.

FIG. 5 is a simplified block diagram of a system for rendering analogaudio signals, in accordance with one embodiment of the presentinvention. As shown in FIG. 5, system 140 includes operating system 152,audio and video data conversion device 150, audio encoder 170, andspeaker 172. Operating system 152 is in communication with audio andvideo data conversion device 150 and audio encoder 170. Operating system152 is the system software responsible for the control and management ofhardware and basic system operations, as well as running applicationsoftware. Exemplary operating system 152 includes Microsoft Windows,Microsoft-DOS, UNIX, Linux, Macintosh Operating System, etc. Includedwithin operating system 152 are streaming driver 166 and audio renderer168. It should be appreciated that streaming driver 166 may include anysuitable driver that supports the processing of streamed data formultimedia devices such as sound cards, TV tuner cards, video graphicscards, etc. Audio renderer 168 includes any suitable renderer thatrenders audio data. An exemplary audio renderer 168 is the MicrosoftDirectSound Audio Renderer that filters and renders audio data.

System 140 may additionally include audio and video data conversiondevice 150. As discussed above, audio and video data conversion device150 may be used to connect the video game console to the computerthrough Universal Serial Bus (USB) such that video and audio from thevideo game console may be viewed, listened, and captured on thecomputer. It should be appreciated that audio and video data conversiondevice 150 is merely an example of one type of connection between avideo game console and computer. Other types of connections may includeaudio cables that connect the video game console to a sound card of thecomputer, video cables that connect the video game console to a videocard of the computer, etc. System 140 additionally includes audioencoder 170 which converts digital audio data to analog audio data andrenders the analog audio data to speaker 172.

As shown in FIG. 5, audio decoder 162 receives analog audio signal 154and converts the analog audio signal to digitized audio data 156 at anaudio capture clock frequency (e.g., 48 KHz, 48.05 KHz, etc.) for outputto USB controller 164. USB controller 164 receives digitized audio data156 and transfers the streaming, digitized audio data to streamingdriver 166 of operating system 152 via USB bus 158. Subsequently,streaming driver 166 processes the audio data and outputs digitizedaudio data 160 to audio renderer 168 for rendering. As a result, audiorenderer 168 sends a buffered, digital audio data to audio encoder 170at an audio rendering clock frequency (e.g., 48.00 KHz). Audio encoder170 then converts the digital audio data to analog audio data forrendering on speaker 172.

FIG. 6 is a more detailed block diagram of the operating system shown inFIG. 5, in accordance with one embodiment of the present invention. Asshown in FIG. 6, operating system 152 includes streaming driver 166 andaudio renderer 168. After streaming driver 166 passes digitized audiodata 160 to audio renderer 168, the audio renderer temporarily storesthe digitized audio data in audio buffer 204 for rendering, which allowsthe audio renderer and an audio encoder that process the audio data atdifferent speeds to operate without being delayed by one another.

With reference to audio buffer 204, FIG. 6 additionally shows fullness208 that indicates the amount of audio data stored in the audio buffer.For example, a large fullness 208 value indicates a large amount ofaudio data stored in audio buffer 204. It should be noted that theamount of audio data stored in audio buffer 204 has a direct correlationwith the delay of audio signal processing. For instance, the larger thefullness 208, the longer the delay of processing the audio signals.Furthermore, as will be explained in more detail below, embodiments ofthe invention use top threshold value 206 and bottom threshold value 210for reducing audio latency by adjusting incrementally the audio datafeed to audio renderer 168 such that the amount of audio data (i.e.,fullness 208) stored in audio buffer 204 is between the top thresholdvalue and the bottom threshold value.

Further, since an embodiment of the invention adjusts the audio datafeed to audio renderer 168, embodiments of the invention may be includedin streaming driver 166. For example, in one embodiment, streamingdriver 166 may additionally include program instructions 212 foradjusting incrementally the audio data feed from the streaming driver toaudio renderer 168. Specifically, as will be explained in more detailbelow, program instructions 212 included in streaming driver 166 maymake the adjustments by decreasing or increasing the audio data feed toaudio renderer 168 such that fullness 208 is between top threshold value206 and bottom threshold value 210.

FIG. 7 is a flowchart diagram of a high level overview of a method forreducing audio latency when executing program instructions forprocessing audio data, in accordance with one embodiment of the presentinvention. Starting in operation 302, an amount of audio data stored inan audio buffer is first determined. Thereafter, in operation 304, theamount of audio data stored in the audio buffer is compared with a topthreshold value and a bottom threshold value. The audio data feed isthen adjusted incrementally in operation 306 such that the amount ofstored audio data is between the top threshold value and the bottomthreshold value.

FIG. 8 is a flowchart diagram of a more detailed method for reducingaudio latency when executing program instructions for processing audiodata, in accordance with one embodiment of the present invention.Starting with operations 402 and 404, a top threshold value and a bottomthreshold value are provided. Top threshold value may include anysuitable value, and bottom threshold value may include any suitablevalue that is less than the top threshold value. In one exemplaryembodiment, top threshold value may be calculated by: $\begin{matrix}{{Top\_ threshold} = {{freq\_ audio}\left( \frac{{Max\_ delay}{\_ time}}{1000} \right)}} & (1.0)\end{matrix}$With regard to Equation (1.0), maximum delay time (i.e., max_delay_time)is a time limit used by embodiments of the invention to assure thataudio latency will be within the maximum delay time during rendering byaudio renderer. An exemplary maximum delay time is 50 ms. Freq_audio isthe standard audio frequency, which may be different from audio captureclock frequency and audio capture clock frequency as discussed above.Exemplary standard audio frequencies include 48 KHz, 44.1 KHz, 32 KHz,etc. Accordingly, in one embodiment, a top threshold value may becalculated if the maximum delay time and the standard audio frequencyare provided. For example, assuming maximum delay time=50 ms andstandard audio frequency=48 KHz, then top threshold is (50ms/1000)*48000, which equals 2400 samples.

Bottom threshold value may include any suitable value that is less thantop threshold value. In one embodiment, bottom threshold value may bederived from top threshold value. For instance, bottom threshold valuemay be calculated by: $\begin{matrix}{{Bottom\_ threshold} = \frac{Top\_ threshold}{3}} & (1.1)\end{matrix}$Accordingly, with reference to the top threshold value of 2400 samplesdiscussed above, the bottom threshold value is simply 2400 samples/3,which equals 800 samples.

Still referring to FIG. 8, the amount of audio data stored in the audiobuffer is then determined in operation 406. The amount of audio data maybe determined by receiving the fullness value directly from audiorenderer. Thereafter, the amount of audio data is compared with the topthreshold value and the bottom threshold value in operation 408. Asshown in operation 410, the top threshold value is compared with theamount of audio data stored in the audio buffer to determine whether thestored amount is greater than the top threshold value. If the amount ofdata stored in audio buffer is greater than top threshold value, thenthe audio data feed from a streaming driver is decreased by anincremental amount in operation 412. Specifically, the streaming driverfirst receives a sample of the audio data and reduces the sample of thereceived audio data by the incremental amount through interpolation. Itshould be appreciated that any suitable interpolation techniques may beapplied. Exemplary interpolation techniques include linear interpolationand non-linear interpolation. Additionally, interpolation may simplyinclude taking the floor or ceiling of the audio data.

Subsequently, the streaming driver outputs the reduced sample of audiodata to the audio renderer. For example, if the incremental amount isspecified as two audio data samples in each 1000 samples, then for every1000 samples received by streaming driver, streaming driver reduces twosamples from the 1000 received samples, and interpolates the 1000received samples to generate 998 samples. Streaming driver then outputsthe 998 samples of audio data to the audio renderer. If the amount ofaudio data stored in the audio buffer is still greater than the topthreshold value after the interpolation, then the audio data is stillfed to the audio renderer faster than the speed at which the audiorenderer can render the audio data. Accordingly, the streaming driverwill further reduce the number of audio data samples to 996. Thereduction will repeat for subsequent samples of the audio data until theamount of audio data buffered in audio renderer is less than or equal tothe top threshold value.

On the other hand, as shown in FIG. 8, if the amount of audio datastored in audio buffer is not greater than top threshold value, thenanother comparison may be conducted in operation 411 to determinewhether the amount of audio data is less than the bottom thresholdvalue. If the amount of audio data is not less than the bottom thresholdvalue, then the amount of audio data is between the top threshold valueand the bottom threshold value, and no further adjustments arenecessary. However, if the amount of audio data is less than the bottomthreshold value, then the audio data feed from the streaming driver isincreased by an incremental amount in operation 414. Specifically, thestreaming driver first receives a sample of the audio data and increasesthe sample of the audio data by the incremental amount throughinterpolation. Subsequently, the streaming driver outputs the increasedsample of the audio data to the audio renderer. For example, if theincremental amount is again specified as two audio data samples in each1000 samples, then for every 1000 samples received by streaming driver,the streaming driver adds two samples to the 1000 received samplesthrough interpolation to generate 1002 samples. Streaming driver thenoutputs the 1002 samples of audio data to the audio renderer. If theamount of audio data stored in the audio buffer is still less than thebottom threshold value after the interpolation, then the audio data isstill fed to audio renderer slower than the speed at which the audiorenderer can render the audio data. Accordingly, the streaming driverwill further increase the number of audio data samples to 1004. Theincrease will repeat for subsequent samples of the audio data until theamount of audio data stored in the audio renderer is greater than orequal to the bottom threshold value.

As discussed above, the incremental amount is the number of samples thatis adjusted each time when the amount of audio data stored in audiobuffer is either larger than the top threshold value or lesser than thebottom threshold vale. Incremental amount may include any suitablevalues. For example, in one embodiment, the incremental amount can becalculated by: $\begin{matrix}{{Incremental\_ amount} = {{original\_ sample}{\_ size}\left( {\frac{{Freq\_}2}{{Freq\_}1} - 1} \right)}} & (1.2)\end{matrix}$Referring to Equation (1.2), original sample size is the number of audiosamples received by streaming driver. As discussed above, audiorendering clock frequency (i.e., Freq_(—)2) is the frequency at which anaudio renderer sends audio data to an audio encoder, and audio captureclock frequency (i.e., Freq_(—)1) is the frequency at which an audiodecoder converts the analog audio signal to digitized audio data. Forexample, assuming that the standard audio frequency (i.e., freq_audio)is 48 KHz and the amount of audio data stored in the audio bufferincreases from 0 to 2400 samples (50 ms delay) during 24 seconds, whichis about a one second delay in eight minutes ((8*60/24)*50 ms=1000 ms=1s), then for each second the amount of audio data increases by 100samples, and Freq_(—)2/Freq_(—)1=(48000+100)/48000=481/480. If originalsample size is 481, then incremental amount=481*(481/480−1)=1. Thus, onesample is reduced for every 481 samples and, as a result, the reducedaudio data is fed to the audio renderer at the same speed as the audiorenderer renders the audio data.

FIG. 9 is a flowchart diagram of a detailed method for implementing anembodiment of the present invention in Microsoft Windows. Starting inoperation 502, the streaming driver creates an application programinterface (API) to receive audio buffer fullness information from anapplication that reads the buffer fullness information from theDirectSound Audio Renderer. Specifically, to create the API, MicrosoftStream Class minidriver defines the following property set GloballyUnique Identifier (GUID) and implements the property set on anIKSPropertySet interface. The property set allows applications to sendcommands to the driver through the IKSPropertySet interface with GUIDPROPSETID_ADAPTEC_PROPERTIES, which is defined as {0x2d2c4cd1, 0xd8c7,0x45ae, {0xb5, 0x4a, 0xee, 0x7c, 0x66, 0xa, 0x82, 0x54} }. Here, theproperty set GUID PROPSETID_ADAPTEC_PROPERTIES implements settingsrelated to specific properties. For example, the following Table Asummarizes an exemplary Property ID that may be implemented in aproperty set and also shows the parameters to be passed to the Set andGet calls. TABLE A Command Property ID Code Set/Get pPropDataADPT_AUDIO_RENDERER_(—) 0x01 set byte FULLNESS

As shown in FIG. 9, after the API is created, the API gets the audiobuffer fullness information from DirectSound Audio Renderer in operation504 through an IAMAudioRendererStats::GetStatParam method. Subsequently,operation 506 shows that a comparison is made to determine whether thefullness is greater than a top threshold value or lesser than a bottomthreshold value. If the fullness is greater than the top threshold valueor lesser than the bottom threshold value, then the application sendsthe fullness information to streaming driver in operation 508 throughthe IKSPropertySet interface with property Set GUID(PROPSETID_ADAPTEC_PROPERTIES) and Property Id(ADPT_AUDIO_RENDERER_FULLNESS). Else, if the fullness is between the topthreshold value and the bottom threshold value, then the fullnessinformation is not sent to the streaming driver.

FIG. 9 further shows that after operation 508, streaming driverincrementally adjusts the audio data feed in operation 510. As discussedabove, the audio data feed to the audio renderer is increased ordecreased incrementally through interpolation. In other words, thestreaming driver shrinks or expands the audio data feed throughinterpolation according to whether the fullness is greater than the topthreshold value or lesser than the bottom threshold value before passingthe audio data to audio renderer. The following Table B is an exemplaryembodiment of program instructions for adjusting incrementally the audiodata feed to reduce audio latency. TABLE B #define ORIGINAL_SAMPLE_SIZE1000 #define ADJUSTED_SIZE 2 #define BOTTOM_THRESHOLD 200 #defineTOP_THRESHOLD 400 WORD wOriginalAudioSamples [ORIGINAL_SAMPLE_SIZE];WORD wAdjustedAudioSamples [ORIGINAL_SAMPLE_SIZE + ADJUSTED_SIZE]; WORDwFullness; WORD wAdjustedSampleSize = ORIGINAL_SAMPLE_SIZE; WORDAdjustAudioSamples ( ) { WORD I, wFloor, wCeiling; DOUBLE dDistance; if(wFullness > TOP_THRESHOLD) { wAdjustedSampleSize −= ADJUSTED_SIZE; }else if (wFullness < BOTTOM_THRESHOLD) { wAdjustedSampleSize +=ADJUSTED_SIZE } if (wAdjustedSampleSize > ORIGINAL_SAMPLE_SIZE +ADJUSTED_SIZE) { wAdjustedSampleSize = ORIGINAL_SAMPLE_SIZE +ADJUSTED_SIZE } else if (wAdjustedSampleSize < ORIGINAL_SAMPLE_SIZE −ADJUSTED_SIZE) { wAdjustedSampleSize = ORIGINAL_SAMPLE_SIZE −ADJUSTED_SIZE } if (wAdjustedSampleSize == ORIGINAL_SAMPLE_SIZE) { For(I = 0; I < wAdjustedSampleSize; I++) { wAdjustedAudioSamples[I] =wOriginalAudioSamples[I]; } } else { for (I = 0; I <wAdjustedSampleSize; I++) { dDistance = (DOUBLE1.0)*I*(ORIGINAL_SAMPLE_SIZE − 1)/ (wAdjustedSampleSize − 1); wFloor =floor(dDistance); wCeiling = ceiling(dDistance);wAdjustedAudioSamples[I] = wOriginalAudioSamples[wFloor]* (1 −(dDistance − wFloor)) + wOriginalAudioSamples[wCeiling]* (dDistance −wFloor); } } return wAdjustedSampleSize; }The algorithms included in Table B are merely exemplary, and manydifferent algorithms may be used to reduce or increase samples of audiodata.

It should be appreciated that the above-described functionality forreducing audio latency may be incorporated in program application storedin memory (e.g., random access memory (RAM), hard disk drives, floppydisks, magnetic tapes, optical discs, etc.) and executed by a processor.For example, the functionality may be provided through the streamingdriver, or the like, having program instructions to perform theabove-described functionality. In one embodiment, streaming driverincludes program instructions for determining an amount of the audiodata stored in an audio buffer and program instructions for comparingthe amount with a top threshold value and a bottom threshold value.Further, program instructions are included for adjusting incrementallyan audio data feed to an audio renderer such that the amount is betweenthe top threshold value and the bottom threshold value.

In sum, the above described invention provides method and system forreducing audio latency when executing program instructions forprocessing audio data. Essentially, to reduce latency, streaming driverfeeds audio data to an audio renderer slower than the speed at which theaudio renderer can render audio data if the amount of audio data storedin the audio buffer is greater than a top threshold value. On the otherhand, streaming driver may be additionally configured to feed audio datato the audio renderer faster than the speed at which the audio renderercan render audio data if the amount of audio data is less than a bottomthreshold value. Thus, by keeping the amount of audio data stored inaudio buffer between the top threshold value and the bottom thresholdvalue, audio latency is reduced or altogether eliminated such that audiois synchronized with video.

C. Graphical User Interface

A video game console display application can be executed on a computerto process, to facilitate real-time video game playing, to control, andto display video and/or audio from the video game console. It should beappreciated that the video game console display application can beintegrated or combined with other software modules, such as the audiorenderer discussed above. The video game console display application canoptimize video display for video gaming to provide rich audio and visualeffects. For example, in one embodiment, the video game console displayapplication can adjust properties of the video before output to adisplay panel. Exemplary video properties include brightness, sharpness,contrast, shape, color, resolution, refresh rate (or vertical scanrate), size, etc. For example, the size of the display area for videocan be enlarged or reduced. Further, a display panel used with computerstypically have higher refresh rates than conventional televisions.Accordingly, in another example, the video game console displayapplication can increase the refresh rate of the video received from thevideo game console, thereby reducing eye strain. In another embodiment,the video game console display application can process the audio data toreduce the audio latency as discussed above to provide a real-timegaming experience. In still another embodiment, the video game consoledisplay application can include controls to allow capture and storage ofvideo and/or audio from the video game console. For example, videodisplays of game configuration settings, best score results, locationsof hidden treasures in a video game, etc. can be stored. The recordedvideo and/or audio may be stored in a memory (e.g., RAM, hard diskdrives, floppy disks, magnetic tapes, optical disks, etc.) of thecomputer.

FIG. 10 is a schematic diagram of a main graphical user interface (GUI)associated with the video game console display application, inaccordance with one embodiment of the present invention. After the videogame console display application is launched on a computer, main windowregion 902 is displayed on a display panel coupled to the computer. Mainwindow region 902 includes a number of GUI components 904-909 forproviding access to various audio and video functionalities. As shown inFIG. 10, main window region 902 includes video game GUI component 904,TV GUI component 905, digital versatile disc (DVD)/video CD (VCD) GUIcomponent 906, videos GUI component 907, photos component 908, and musiccomponent 909.

Video game GUI component 904 provides access to the video and/or audioreceived from the video game console. In response to video game GUIcomponent 904 being selected (e.g., by clicking with a mouse, inputthrough a keyboard, or input through any suitable input devices),another region is generated that displays the video received from thevideo game console. TV GUI component 905 provides access to videoreceived from a TV tuner, and a selection of the TV GUI componentgenerates another region that displays the video from the TV tuner. Inaddition to video and/or audio received from the audio and video dataconversion device, main window region 902 allows additional access tovideo and/or audio from additional sources, such as video players (e.g.,DVD/VCD players), memory of the computer, external hard drives, etc. Forexample, DVD/VCD GUI component 906 allows access to video and/or audioreceived from DVD/VCD players, videos GUI component 907 allows access tovideo stored in a memory of the computer, photos GUI component 908allows access to photos stored in the memory of the computer, and musicGUI component 909 allows access to audio files stored in the memory ofthe computer.

Any number of suitable layouts can be designed for region and GUIcomponent layouts illustrated above as FIG. 10 does not represent allpossible layout options available. The displayable appearance of theregions and GUI components can be defined by any suitable geometricshape (e.g., rectangle, square, circle, triangle, etc.), alphanumericcharacter (e.g., A,v,t,Q,1,9,10, etc.), symbol (e.g., $,*,@,α,

,¤,♥etc.), shading, pattern (e.g., solid, hatch, stripes, dots, etc.),and color. Furthermore, for example, TV GUI component 905 of FIG. 10, orany other regions or GUI components, may be omitted or dynamicallyassigned. It should also be appreciated that the regions and GUIcomponents can be fixed or customizable. In addition, the computer mayhave a fixed set of layouts, utilize a defined protocol or language todefine a layout, or an external structure can be reported to thecomputer that defines a layout. Finally, selecting a region or GUIcomponent of GUI triggers code to cause the functionality describedherein.

In summary, the above described embodiments provide methods, systems,and GUIs for reducing audio latency and for displaying video from avideo game console. Essentially, the audio and video data conversiondevice connects a video game console to a computer and enables users toplay video games using the resources (e.g., display panel, speakers,etc.) available to the computer with real-time gaming effect, similar towhen the video game console is connected to a regular television set.The embodiments described above can actually keep the latency betweenaudio/video effects and user's input actions through the video gameconsole to less than 100 milliseconds, which is negligible to humans.

With the above embodiments in mind, it should be understood that theinvention may employ various computer-implemented operations involvingdata stored in computer systems. These operations are those requiringphysical manipulation of physical quantities. Usually, though notnecessarily, these quantities take the form of electrical or magneticsignals capable of being stored, transferred, combined, compared, andotherwise manipulated. Further, the manipulations performed are oftenreferred to in terms, such as producing, identifying, determining, orcomparing.

The invention can also be embodied as computer readable code on acomputer readable medium. The computer readable medium is any datastorage device that can store data which can be thereafter read by acomputer system. The computer readable medium also includes anelectromagnetic carrier wave in which the computer code is embodied.Examples of the computer readable medium include hard drives, networkattached storage (NAS), read-only memory, random-access memory, CD-ROMs,CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical datastorage devices. The computer readable medium can also be distributedover a network coupled computer system so that the computer readablecode is stored and executed in a distributed fashion.

Any of the operations described herein that form part of the inventionare useful machine operations. The invention also relates to a device oran apparatus for performing these operations. The apparatus may bespecially constructed for the required purposes, or it may be a generalpurpose computer selectively activated or configured by a computerprogram stored in the computer. In particular, various general purposemachines may be used with computer programs written in accordance withthe teachings herein, or it may be more convenient to construct a morespecialized apparatus to perform the required operations.

The above described invention may be practiced with other computersystem configurations including hand-held devices, microprocessorsystems, microprocessor-based or programmable consumer electronics,minicomputers, mainframe computers and the like. Although the foregoinginvention has been described in some detail for purposes of clarity ofunderstanding, it will be apparent that certain changes andmodifications may be practiced within the scope of the appended claims.Accordingly, the present embodiments are to be considered asillustrative and not restrictive, and the invention is not to be limitedto the details given herein, but may be modified within the scope andequivalents of the appended claims. In the claims, elements and/or stepsdo not imply any particular order of operation, unless explicitly statedin the claims.

1. A system for displaying video from a video game console, comprising:an audio and video data conversion device in communication with thevideo game console; a computer in communication with the audio and videodata conversion device; and a display panel in communication with thecomputer, the display panel being capable of displaying the video. 2.The system of claim 1, wherein the audio and video data conversiondevice includes, an audio and video processing circuitry.
 3. The systemof claim 2, wherein the audio and video data conversion device furtherincludes, a device controller in communication with the audio and videoprocessing circuitry, the device controller enabling communication withthe computer.
 4. The system of claim 2, wherein the audio and videoprocessing circuitry includes, a video decoder configured to receive thevideo from the video game console; and an audio decoder in communicationwith the video decoder, the audio decoder being configured to receiveaudio from the video game console.
 5. The system of claim 2, wherein theaudio and video data conversion device further includes, a televisiontuner in communication with the audio and video processing circuitry. 6.The system of claim 1, wherein the computer includes a memory, thememory being configured to store the video from the video game console.7. The system of claim 6, wherein the memory is further configured tostore audio received from the video game console.
 8. The system of claim1, further comprising: a speaker in communication with the computer, thespeaker being capable of outputting audio outputted from the video gameconsole.
 9. The system of claim 1, wherein a graphical user interface(GUI) for a video game console display application is rendered on thedisplay panel, the GUI including, a first GUI component providing accessto the video from the video game console; and a first region displayingthe video, the first region being generated in response to a selectionof the first GUI component.
 10. The system of claim 9, wherein the GUIfurther includes, a second GUI component providing access to videoreceived from a television tuner.
 11. The system of claim 10, whereinthe GUI further includes, a third region displaying the video receivedfrom the television tuner, the third region being generated in responseto a selection of the second GUI component.
 12. The system of claim 9,wherein the GUI further includes, a third GUI component providing accessto video received from a video player.
 13. A method for displaying videofrom a video game console, comprising: receiving video at a computerfrom an audio and video data conversion device, the video being fed tothe audio and video conversion device by the video game console; andoutputting the video to a display panel.
 14. The method of claim 13,further comprising: receiving audio at the computer from the audio andvideo data conversion device, the audio being fed to the audio and videoconversion device by the video game console; and outputting the audio toa speaker.
 15. The method of claim 13, further comprising: adjusting aproperty of the video.
 16. The method of claim 15, wherein the propertyis defined by one or more of brightness, sharpness, contrast, shape,color, resolution, refresh rate, and size.
 17. The method of claim 13,further comprising: recording the video.
 18. The method of claim 13,wherein the audio and video data conversion device is external to thecomputer.
 19. A method for displaying video from a video game console,comprising method operations of: converting analog video from the videogame console to digitized video; and outputting the digitized video to acomputer.
 20. The method of claim 19, further comprising: convertinganalog audio from the video game console to digitized audio; andoutputting the digitized audio to the computer.